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Automated  Cartographic  Feature  Attribution  Using  Panchromatic  and 

Hyperspectral  Imagery 
DARPA/APGD  Yearly  Report  1997-1998 


1.  Introduction 

This  report  summerizes  the  primary  accomplishments  made  during  the  first  year  of  the  Defense  Advanced 
Research  Projects  Agency  (DARPA)  Automated  Population  of  Geographic  Databases  (APGD)  program. 
Prior  to  this  contract,  late  in  1995,  we  organized  a  hyperspectral  data  acquisition  using  the  Naval  Research 
Laboratory’s  (NRL)  Hyperspectral  Digital  Imagery  Collection  Experiment  (HYDICE)  sensor  system  over 
Fort  Hood,  TX.  This  acquisition  resulted  in  hyperspectral  data  with  a  nominal  2-meter  ground  sample  dis¬ 
tance  collected  with  210  spectral  samples  per  pixel.  This  data  formed  the  basis  of  our  program  of  research 
under  APGD. 

Section  2.  describes  experiments  in  the  block  adjustment  of  linear  pushbroom  sensor  imagery  incorporat¬ 
ing  object-space  straight  line  constraints.  Both  polynomial  and  inter polative  platform  models  were  tested; 
the  polynomial  model  generally  performed  better  than  the  interpolative  model  without  lines,  but  not  as 
well  as  the  interpolative  model  with  straight  line  constraints.  The  greater  flexibility  of  the  interpolative 
model  makes  it  better  able  to  describe  complex  platform  motion  and  to  use  the  geometric  strength  given 
by  the  straight  line  constraints,  at  the  expense  of  increased  sensitivity  to  uneven  point  distributions  or  bad 
points  [McGlone,  1998]. 

Section  3.  describes  current  quantitative  classification  results  for  man-made  and  natural  materials  us¬ 
ing  14  surface  material  classes  over  selected  test  areas  within  Fort  Hood.  We  discuss  the  issues  encoun¬ 
tered  in  radiometric  effects  due  to  changing  solar  illumination  and  atmospheric  conditions  during  the 
acquisition[Ford  et  al.,  1998]. 

1.1  Background  for  CMU  APGD  Research  Project 

A  major  focus  of  this  research  during  the  last  5  years  has  been  the  use  of  multispectral  imagery  to  generate 
surface  material  maps.  Surface  material  information  is  of  interest  to  us  both  for  cartographic  feature 
extraction  (CFE),  to  generate  feature  hypotheses  or  to  refine  features  generated  by  other  CFE  systems, 
and  for  visual  simulation,  to  select  realistic  visual  textures. 

Some  of  our  previous  research  in  fusion  has  involved  the  combination  of  surface  material  information 
with  high-resolution  stereo  elevation  data  to  produce  more  refined  surface  material  maps  and  to  aid  in 
distinguishing  man-made  structures  within  the  scene  [Ford  and  McKeown,  1992a],  [Ford  and  McKeown, 
1992b].  We  are  currently  studying  the  combination  of  surface  material  maps  with  automatically  extracted 
buildings  and  roads  for  editing  and  verification. 

One  important  limitation  in  this  work  has  been  the  limited  spatial  resolution  (8-20  meters)  of  available 
multispectral  imagery.  Since  we  require  the  precise  delineation  of  object  boundaries  and  the  attribution 
of  surface  materials  for  small  regions,  such  as  those  found  within  urban  areas,  we  typically  work  with 
panchromatic  imagery  with  ground  sample  distances  (GSD)  of  0.3  to  1.0  meters.  Combining  classification 
results  from  multispectral  imagery  with  much  coarser  spatial  resolution  imposes  limitations  on  the  utility 
of  the  fusion. 

In  order  to  obtain  data  with  higher  spatial  and  spectral  resolution  we  organized  a  hyperspectral  data 
acquisition  using  the  NRL  HYDICE  sensor.  This  acquisition,  in  October  1995,  resulted  in  hyperspectral 
data  with  a  nominal  2-meter  GSD  collected  with  210  spectral  samples  per  pixel.  The  acquisition  covered 
56  km2  over  Fort  Hood,  TX,  a  site  that  has  been  used  extensively  in  the  DARPA  Image  Understanding 
community  to  support  experiments  in  semi-automated  and  automated  CFE. 
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1.2  Geometric  and  radiometric  issues 


Two  major  problems,  radiometry  and  geopositioning,  must  be  addressed  in  order  to  effectively  use  the 
HYDICE  imagery  in  conjunction  with  other  types  of  imagery  (panchromatic,  radar,  lidar)  used  for  carto¬ 
graphic  feature  extraction. 

The  radiometric  problems  arise  from  the  effects  of  changing  atmospheric  conditions  and  solar  illumina¬ 
tion  on  the  ground-reflected  spectral  radiance  collected  by  the  hyperspectral  sensor.  In  order  to  compare 
surface  material  properties  between  flightlines  and  to  use  spectral  field  measurements  of  surface  materials 
for  spectral  analysis,  atmospheric  corrections  are  applied  to  convert  HYDICE  radiance  imagery  to  apparent 
reflectance. 

Geometric  issues  are  due  to  the  dynamic  nature  of  the  image  acquisition  process  and  to  the  weak  geo¬ 
metric  configuration  of  the  sensor,  as  discussed  below,  in  “Registration  of  HYDICE  Imagery.”  For  large- 
scale  mapping  or  more  standard  remote  sensing  applications,  accurate  positioning  has  been  less  important; 
however,  to  meet  our  goals  of  fusing  surface  material  regions  with  features  derived  from  our  road  and 
building  extraction  systems  for  high-resolution  site  modeling,  extremely  accurate  absolute  geopositioning 
and  relative  registration  between  images  must  be  established. 

1.3  HYDICE  Data  acquisition 

The  collection  of  data  at  Fort  Hood  included  both  airborne  imagery  and  ground  truth  measurements.  The 
image  acquisition  included  hyperspectral  imagery  collected  by  the  HYDICE  sensor  system  and  natural 
color  film  shot  by  a  KS-87  frame  reconnaissance  camera.  The  spectral  range  of  the  HYDICE  sensor  ex¬ 
tends  from  the  visible  to  the  short  wave  infrared  (400-2500  nm)  regions,  divided  into  210  channels  with 
nominal  10  nm  bandwidths. 

Nine  HYDICE  flightlines,  each  640  meters  wide  (cross-track)  and  12.6  km  long  (along-track) ,  were  flown 
over  Fort  Hood’s  motor  pool,  barracks,  and  main  complex  areas  from  an  altitude  of  approximately  4,000  me¬ 
ters  above  ground  level.  After  each  flightline,  the  HYDICE  sensor  was  flown  over  and  imaged  a  six-step  (2, 
4,  8,  16,  32  and  64  percent)  gray-scale  panel,  providing  in-scene  radiometric  calibration  measurements  for 
each  flightline.  Prior  to  the  start  of  the  HYDICE  flight  collection,  several  ground  spectral  measurements 
were  made  for  each  gray  level  panel  in  an  attempt  to  characterize  its  mean  spectral  reflectance  curve.  A 
more  detailed  description  of  the  HYDICE  sensor  system,  Fort  Hood  image  acquisition  and  ground  truthing 
activities,  can  be  found  in  [Ford  et  al.,  1997a]. 


2.  Block  Adjustment  of  Linear  Pushbroom  Imagery  with  Geometric  Con¬ 
straints 

Linear  pushbroom  imaging  sensors  have  become  widely  used  within  the  last  few  years  as  a  cost-effective 
means  to  obtain  aerial  digital  imagery.  Linear  arrays  are  less  expensive  to  fabricate  than  area  arrays  and 
require  no  moving  parts,  unlike  scanners  or  panoramic  cameras. 

The  main  drawback  to  the  use  of  linear  pushbroom  sensors  is  their  weak  geometry;  each  image  line  is, 
in  effect,  an  independent  one-dimensional  image.  Resecting  an  individual  line  is  an  indeterminate  prob¬ 
lem,  while  using  the  whole  image  requires  that  a  model  of  the  platform  motion  as  a  function  of  time  be 
solved.  The  availability  of  accurate  navigation  data,  such  as  differential  GPS  and  high-resolution  Inertial 
Navigation  System  (INS)  sensors,  has  alleviated  this  problem  somewhat.  Another  approach  has  been 
to  use  multiple  linear  arrays,  pointing  in  the  nadir  and  off-nadir  directions,  to  improve  the  geometry  by 
obtaining  a  wider  cone  of  rays  from  the  same  sensor  position  and  also  to  allow  stereo  viewing.  Examples 
of  this  configuration  include  the  Monocular  Electro-Optical  Stereo  Scanner  (MEOSS)  and  the  Modular 
Optoelectronic  Multispectral  Stereo  (MOMS-02)  sensors  [Ohloff,  1995]. 

This  annual  report  describes  block  adjustment  experiments  using  linear  pushbroom  and  frame  imagery. 
This  solution  differs  somewhat  from  current  practice  in  that  only  limited  navigation  information  (nominal 
GPS  positions)  was  available,  due  to  equipment  problems  during  data  acquisition.  While  these  experiments 
may  not  appear  directly  relevant  to  the  state  of  the  art,  they  are  important  for  several  reasons: 
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•  From  a  practical  standpoint,  navigation  equipment  sometimes  fails  and  reflights  are  not  always  an  option. 
Alternative  methods  of  positioning  can  be  necessary. 

•  Navigation  information  may  not  provide  sufficient  absolute  positioning  accuracy  due  to  the  inherent  errors 
of  the  navigation  sensors.  As  higher-resolution  digital  imagery  becomes  available,  the  positioning  require¬ 
ments  will  become  greater.  These  requirements  may  be  met  by  improving  navigation  sensors  or  by  adding 
additional  information  to  the  solution;  the  choice  is  a  matter  of  the  economics  of  the  particular  system.  A 
related  issue  is  the  discrete  nature  of  the  positioning  information,  which  is  available  only  at  intervals.  Sen¬ 
sor  behavior  between  readings  must  be  interpolated,  on  the  assumption  that  the  characteristics  of  the  plat¬ 
form  motion  do  not  change.  Adding  external  information  between  reference  points  will  make  this  interpo¬ 
lation  more  valid. 

•  An  increasing  amount  of  work  is  currently  being  devoted  to  the  fusion  of  imagery  from  different  sensors, 
taken  at  different  times.  While  each  set  of  imagery  may  be  positioned  to  some  level  of  absolute  accuracy, 
the  accumulated  relative  error  between  image  sets  may  make  fusion  difficult.  A  simultaneous  adjustment 
of  all  the  imagery,  with  additional  information  in  the  form  of  tie  points  and  geometric  constraints,  is  nec¬ 
essary  in  such  a  case. 

•  Positioning  from  navigation  data  alone  is  inherently  open-loop,  in  that  there  is  no  external  verification  or 
redundant  determination  of  the  positions.  For  this  reason,  most  block  solutions  based  on  navigation  infor¬ 
mation  use  a  few  control  points  to  establish  the  datum  for  verification  purposes.  This  level  of  redundancy 
is  adequate  for  well-calibrated  photogrammetric  systems;  however,  for  experimental  remote  sensing  systems 
such  as  the  HYDICE,  which  are  not  designed  as  mapping  systems,  the  system  calibration  may  not  be  suf¬ 
ficient  for  reasonable  positioning  accuracy.  Including  external  data  can  provide  assurances  on  the  quality 
of  the  results  and  also  insights  into  any  calibration  deficiencies. 

The  main  topics  in  these  experiments  were  evaluating  the  differences  between  polynomial  and  inter¬ 
polate  platform  models  and  evaluating  the  use  of  geometric  information,  straight  lines  in  the  scene,  to 
improve  the  block  adjustment. 

Our  interest  in  HYDICE  positioning  is  driven  by  two  main  goals.  First,  we  want  to  generate  high- 
resolution  surface  material  maps,  for  densification  of  land-cover  information  and  for  realistic  material 
rendering  in  visual  simulation  databases  [Ford  et  al. ,  1998].  Our  second  goal  is  to  support  work  in  the 
fusion  of  disparate  types  of  image  information,  in  order  to  improve  cartographic  feature  extraction  [Ford 
and  McKeown,  1992b;  Ford  et  al,  1997b].  While  some  amount  of  mis-registration  can  be  corrected  by 
local  refinement  during  the  fusion  process,  an  inaccurate  initial  registration  greatly  increases  the  amount  of 
down-stream  work  required  and  may  adversely  impact  the  final  quality  of  the  fusion  results. 

2.1  Related  work 

Most  recent  work  on  the  orientation  of  linear  pushbroom  sensors  has  been  focused  on  satellite  sensors, 
especially  SPOT  [Kratky,  1989]  and  MOMS-02  [Ohloff,  1995].  This  differs  from  the  airborne  problem  in 
that  the  platform  motion  is  smoother  and  is  determined  by  the  orbital  parameters. 

In  airborne  work,  [Heipke  et  al. ,  1996]  summarize  work  on  the  airborne  test  data  from  the  MEOSS 
sensor,  which  uses  three  linear  arrays,  (forward-,  nadir-,  and  backward-looking),  to  obtain  stronger  ge¬ 
ometry.  They  use  an  interpolative  platform  model  with  full  navigation  information  and  a  large  number  of 
automatically-generated  tie  points  in  a  block  adjustment  of  four  image  strips. 

Most  work  on  using  geometric  constraints  for  orientation  has  been  done  using  frame  imagery  [Mikhail, 
1993],  applying  projective  geometry  to  relate  image-  and  object-space  lines.  The  research  described  in 
this  annual  report  is  most  closely  related  to  [McGlone  and  Mikhail,  1981;  McGlone  and  Mikhail,  1982; 
McGlone  and  Mikhail,  1985],  which  applied  straight-line  constraints  in  the  block  adjustment  of  airborne 
multispectral  scanner  data,  and  [Paderes  et  al.,  1984],  which  used  lines  in  the  rectification  of  SPOT  im¬ 
agery. 
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2.2  The  HYDICE  sensor  geometry 

HYperspectral  Digital  Imagery  Collection  Experiment  (HYDICE)  is  an  experimental  210-channel  hyper- 
spectral  imaging  system  developed  by  the  Naval  Research  Laboratory.  The  HYDICE  sensor  is  geomet¬ 
rically  a  linear  pushbroom  sensor  320  pixels  wide;  each  pixel  has  an  instantaneous  field  of  view  of  0.5 
milliradians,  giving  a  total  field  of  view  of  approximately  9  degrees.  Physically,  the  sensor  is  an  area  array, 
with  each  row  of  the  array  producing  one  band  of  the  image  by  imaging  the  incident  energy  from  a  different 
portion  of  the  spectrum.  The  spectral  range  of  the  HYDICE  sensor  extends  from  the  visible  to  the  short 
wave  infrared  regions  (400  to  2500  nm),  divided  into  210  channels.  The  channel  bandwidths  range  from  7.6 
to  14.9  nm,  depending  on  the  channel  location  in  the  electromagnetic  spectrum. 

Ancillary  navigation  and  environmental  information  is  recorded  during  the  acquisition  of  HYDICE  im¬ 
agery.  This  includes  INS  and  GPS  position  and  orientation  data,  flight  stabilization  platform  angles,  and 
instrument  engineering  measurements.  More  detailed  descriptions  of  the  HYDICE  sensor  system  can  be 
found  elsewhere  [Kappus  et  al.,  1996]. 

Unfortunately,  equipment  failures  during  flight  and  some  problems  in  system  integration  resulted  in 
most  of  the  navigation  data  being  unusable.  Turbulent  atmospheric  conditions,  unavoidable  due  to  sensor 
scheduling  constraints,  also  degraded  the  geometry  of  the  imagery. 

2.3  Mathematical  model 

The  mathematical  model  has  several  different  parts;  the  sensor  model,  which  describes  the  imaging  ge¬ 
ometry  of  the  linear  pushbroom  sensor,  the  platform  model,  a  representation  of  the  aircraft  position  and 
orientation  with  respect  to  time,  and  the  block  adjustment  incorporating  the  geometric  (straight  line) 
constraints.  This  section  discusses  each  aspect  of  the  mathematical  model. 

2.3.1  Linear  pushbroom  sensor  model 

A  linear  pushbroom  sensor  can  be  thought  of  as  a  frame  sensor  with  only  one  line  in  the  x,  or  flight  line, 
direction.  The  collinearity  equations,  modified  for  use  with  linear  pushbroom  imagery,  are  [McGlone,  1996]: 
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where  the  x  coordinate  is  0,  y  is  the  image  coordinate  and  y0  is  the  principal  point  along  the  sensor,  / 
is  the  focal  length,  and  XP,YP,  Zp  are  cartesian  world  coordinates  of  the  point.  The  position  parameters, 
Xc,  Yc,  Zc,  and  the  angular  orientation  parameters  u,  <j>,  k,  (which  determine  the  orientation  matrix  M3)3) 
are  given  by  the  platform  model  as  functions  of  time,  or  equivalently,  of  line  number. 

Not  all  of  the  six  orientation  parameters  can  usually  be  recovered  in  a  resection  solution,  due  to  the 
linear  sensor  geometry.  The  <f>  (pitch)  angle  is  highly  correlated  with  position  along  the  flight  line,  while 
the  narrow  field-of-view  and  lack  of  terrain  relief  means  that  the  u>  (roll)  angle  is  correlated  with  the  cross- 
strip  position.  Without  external  information,  such  as  angles  or  positions  from  navigation  sensors,  the  u> 
and  <f>  parameters  must  be  held  to  0  in  the  adjustment. 


2.3.2  Platform  model. 

The  platform  model  describes  the  behavior  of  the  orientation  parameters  as  a  function  of  time  or  line 
number.  Two  different  models  were  studied  in  this  work,  the  polynomial  model  and  the  interpolative 
model. 
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Polynomial  platform  model.  In  the  polynomial  platform  model,  the  value  of  each  parameter  (Xc,  Yc,  Zc, 
at  a  particular  line  is  written  as  a  polynomial  function  of  line  number  x.  The  block  adjustment 
solution  determines  the  polynomial  coefficients,  instead  of  the  parameters  themselves. 

To  model  complex  platform  motions  over  a  long  period  of  time  would  require  high-order  polynomials, 
which  could  lead  to  unstable  solutions.  Instead,  the  flight  line  is  divided  into  sections,  with  each  section 
having  its  own  set  of  lower-order  polynomials  (in  this  case,  cubic) .  Continuity  constraints  on  the  orienta¬ 
tion  parameters  at  the  section  boundaries  ensure  that  calculated  ground  positions  are  continuous  across 
the  boundary. 

Interpolative  platform  model  The  interpolative  model  stores  the  orientation  parameters  of  reference 
lines  at  regular  intervals,  then  calculates  the  parameters  of  intervening  image  lines  by  polynomial  interpo¬ 
lation.  In  this  case,  two  reference  lines  on  either  side  of  the  line  of  interest  are  used,  resulting  in  a  cubic 
interpolation  polynomial  [Press  et  al. ,  1989]. 

The  interpolative  model  has  become  more  widely  used  in  recent  years  since  navigation  data  are  usually 
available  to  specify  the  parameters  of  the  reference  lines.  When  no  navigation  data  are  available,  as  is 
the  case  with  this  data  set,  the  parameters  of  each  reference  line  must  be  solved  in  the  adjustment.  If  no 
control  or  tie  points  are  within  the  interpolation  range  of  a  particular  reference  line,  the  parameters  of  that 
line  will  not  be  adjusted.  A  continuity  constraint  is  applied  between  reference  lines  to  ensure  that  all  lines 
have  consistent  parameters. 

2.3.3  Block  adjustment  procedure 

The  bundle  block  adjustment  is  performed  using  an  object-oriented  photogrammetry  package  [McGlone, 
1995]  that  allows  the  use  of  images  with  different  geometries  and  the  rigorous  incorporation  of  geometric 
constraints. 

2.3.4  Straight-line  geometric  constraints 

Straight-line  constraints  effectively  provide  a  tie  point  in  each  image  line,  particularly  important  in  this 
case  where  no  navigation  data  are  available  and  the  imagery  is  severely  deformed.  The  straight  lines  are 
easy  to  obtain  interactively  from  the  imagery,  and  do  not  require  field  surveying  or  additional  navigational 
equipment. 

The  equation  of  a  straight  line  is  written  as  [Mikhail,  1993]: 

a(X-Xo)  +  P(Y-Yo)  +  y{Z-Zo)  =  0  (2) 

The  line  equation  has  six  parameters,  4  of  which  are  independent.  The  direction  of  the  line  is  represented 
by  the  direction  cosines  a ,  / 3 , 7,  while  the  location  is  fixed  by  specifying  the  coordinates  of  a  reference  point 
on  the  line,  Xo,  Vo,  Zo,  and  constraining  that  point  to  be  the  point  on  the  line  closest  to  the  origin. 

Two  additional  constraint  equations  are  required: 

a2  +  P2  +  y2  =  1  (3) 

Xo  01  +  Yo  P  +  Zo  7  =  0 

The  first  constraint  normalizes  the  direction  cosines,  while  the  second  constraint  equation  is  necessary  to 
give  a  unique  definition  of  the  reference  point. 

Saying  that  the  projection  of  an  image  point,  27,  y,-,  lies  on  a  given  line  in  object  space  is  equivalent  to 
saying  that  the  vector  (image  ray)  from  the  perspective  center,  XC,YC,  Zc,  through  the  image  point,  with 
direction  cosines  a,-,  7,  is  coplanar  with  the  line  vector  a,  P, 7  through  point  X0,  Y0,  Z0.  This  is  expressed 

by  the  scalar  triple  product: 

Xo  —  Xc  Yo-Yc  Zo-Zc 

a  P  7=0  (4) 

<*i  Pi  7 * 
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An  important  consideration  in  the  use  of  constraints  is  efficiency.  This  has  two  aspects;  reducing  the 
number  of  parameters  involved,  and  formulating  the  equations  to  allow  the  most  efficient  normal  equation 
structure.  The  straight  line  constraint  is  written  so  that  point  object  space  coordinates  are  not  explicitly 
referenced  in  the  equations,  thereby  reducing  the  number  of  total  number  of  parameters  in  the  solution. 

This  also  means  that  corresponding  points  do  not  have  to  be  identified  and  measured  on  all  images,  thereby 
simplifying  the  measurement  process. 

2.4  Experimental  plan 

2.4.1  Source  imagery 

Three  sets  of  imagery  are  available  over  Ft.  Hood  and  will  be  used  in  the  final  block  adjustment  of  the 
HYDICE  imagery. 

•  The  HYDICE  imagery,  collected  in  nine  sidelapping  flight  lines  with  a  ground  sample  distance  (GSD)  of 
2-meters. 

•  KS-87  color  frame  imagery,  also  collected  on  the  HYDICE  flights.  The  KS-87  is  an  uncalibrated  frame  re¬ 
connaissance  camera  with  a  6-inch  focal  length  and  a  5-inch  format.  The  imagery  was  scanned  at  a  1-meter 
GSD. 

•  The  Research  And  Development  for  Image  Understanding  Systems  (RADIUS)  Ft.  Hood  imagery.  These 
are  about  40  nadir  and  oblique  images,  taken  with  a  frame  mapping  camera  and  scanned  at  a  GSD  of  0.3- 
meters  for  the  vertical  images.  These  images  have  been  previously  block  adjusted  using  surveyed  ground 
control,  and  provide  the  basic  geometric  strength  for  the  adjustment. 

The  control  points  for  the  adjustment  were  originally  surveyed  for  the  adjustment  of  the  RADIUS  im¬ 
ages.  Tie  points  are  measured  between  all  images. 

2.4.2  Experimental  data  set 

For  the  purposes  of  this  annual  report,  a  small  sub-block  of  the  available  data  is  being  used.  The  sub-block 
includes  two  sidelapping  1,280-line  HYDICE  images,  four  KS-87  images,  and  four  RADIUS  vertical  images. 
Tie  points  between  the  HYDICE  images  and  the  frame  images  were  established  by  manual  measurement, 
with  all  tie  points  being  measured  on  at  least  two  frame  images.  Straight  lines  also  were  measured  manu¬ 
ally  on  at  least  two  frame  images.  The  two  HYDICE  images  used  are  shown  in  Figure  1.  Tie  points  for  the 
heavy  density  case  (described  below)  are  shown  as  diamonds  while  check  points  are  shown  as  crosses.  The 
straight  lines  used  in  the  solution  also  are  shown. 

Three  levels  of  tie  point  density  were  established,  as  shown  in  Figure  2  and  Table  1. 

The  same  37  check  points  shown  in  Figures  1  and  2(d)  were  used  for  each  experiment.  Check  points  that 
appear  on  both  HYDICE  images  are  counted  twice,  as  they  are  treated  independently. 

All  measured  object-space  straight  lines  were  horizontal  and  were  constrained  to  be  horizontal. 

2.4.3  Evaluation  procedure 

Evaluation  was  done  by  comparing  the  calculated  world  X,Y  coordinates  of  the  check  points  against  the 
values  using  the  frame  images.  No  evaluation  was  done  on  the  Z  coordinate,  since  the  HYDICE  sensor  has 
a  very  narrow  field-of-view  (9  degrees)  and  elevation  recovery  is  weak.  For  this  reason,  the  Z  coordinates  of 
the  check  points  were  held  fixed  in  the  solution,  and  points  that  appeared  on  both  HYDICE  images  were 
evaluated  as  two  separate  points.  Deviations  in  the  X,  Y,  and  XY  coordinates  were  calculated  in  a  local 
vertical  coordinate  system,  with  X  being  east  and  Y  north.  In  this  case,  X  also  corresponds  to  the  along- 
strip  direction  and  Y  to  the  cross-strip  direction. 

In  order  to  gain  a  better  understanding  of  the  characteristics  of  the  solution,  three  different  statistics 
were  calculated:  the  median  absolute  deviation,  the  root-mean-square  (RMS)  deviation,  and  the  maximum 
absolute  deviation.  Since  the  RMS  statistic  is  extremely  sensitive  to  large  outliers,  the  median  and  maxi¬ 
mum  statistics  are  used  to  give  a  better  sense  of  the  distribution. 
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(a)  4.3.  (b)  5.3. 

Figure  1:  HYDICE  test  images,  with  tie  points  (diamonds),  check  points  (crosses),  and  constrained  lines 
North  is  to  the  left. 
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Table  1:  Point  test  cases. 


Case 

Figure 

Pts  on  4_3 

Pts  on  5_3 

Pts  on  both 

1  (heavy) 

2(a) 

18 

17 

8 

2  (medium) 

2(b) 

14 

12 

5 

3  (sparse) 

2(c) 

6 

6 

3 

(c)  Point  case  3  (sparse). 

Figure  2:  Point  test  cases,  check  points,  and  image  coverages. 


2.5  Block  Adjustment 

The  results  of  the  evaluation  runs  are  given  in  Table  2  and  graphically  in  Figure  3.  The  interpolative 
model  solution  for  the  sparse  point  case  (3)  with  no  lines  and  32-line  spacing  did  not  converge,  due  to  weak 
geometry  with  the  reduced  number  of  points,  so  no  results  are  given. 

We  rely  mostly  on  the  median  statistics  in  analyzing  the  results,  due  to  the  characteristics  of  the  check¬ 
point  errors.  The  test  runs  show  that  there  are  often  one  or  two  very  large  check-point  deviations,  not 
representative  of  the  rest  of  the  points.  The  RMS  is  greatly  affected  by  these  large  values,  as  opposed  to 
the  median,  which  gives  a  better  sense  of  how  most  of  the  points  behaved. 

Polynomial  vs.  interpolative  platform  models.  The  motion  of  an  airborne  platform  can  be  incred¬ 
ibly  complex,  with  its  characteristics  changing  during  flight.  For  instance,  compare  the  differences  in  de¬ 
formation  between  images  4.3  (Figure  1(a))  and  5.3  (Figure  1(b)),  from  adjacent  flight  lines;  in  particular, 
straight  roads  and  buildings  are  much  more  severely  deformed  in  5_3  than  in  4_3.  Whether  it  is  derived 
from  navigation  data,  by  a  resection  solution,  or  a  combination  of  both,  the  platform  model  must  meet 
a  set  of  contradictory  requirements.  It  must  have  enough  degrees  of  freedom  to  model  the  actual  motion 
with  high  fidelity,  while  too  many  degrees  of  freedom  may  result  in  an  unstable  solution  susceptible  to  bad 
measurements.  Insufficient  degrees  of  freedom  will  result  in  aliasing,  with  unpredictable  results  between 
control  points. 
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Table  2:  Check-point  error  (meters)  for  HYDICE  images. 


Table  3:  Check-point  error  (meters)  on  each  HYDICE  image,  using  interpolative  model. 


Image 

pt 

case 

Lines 

Ref.  line 
spacing 

X 

Y 

XY  1 

Med 

RMS 

Max 

Med 

RMS 

Max 

Med 

RMS 

Max 

4.3 

1 

Y 

4.9 

■H 

3.4 

5.9 

6.0 

■a 

9.8 

5.3 

1 

Y 

■91 

4.6 

mti 

3.9 

21.7 

5.2 

WM 

22.5 

4.3 

2 

Y 

mm 

im 

■9 

■g 

10.9 

5.6 

m 

EEfli 

2 

Y 

■£■ 

mm\ 

E9 

21.7 

4.7 

DD 

E1M 

3 

Y 

mm 

8.4 

■BH 

4.8 

■Egjjl 

14.7 

8.2 

3 

Y 

— 

5.1 

■fiHi 

7.6 

IPIkI 

56.2 

8.1 

(a)  Heavy  point  density  (case  1).  (b)  Sparse  point  density  (case  3). 

Figure  4:  Check  point  errors  for  interpolative  model  with  line  constraints. 


For  this  data  set,  the  polynomial  model  generally  performed  better  than  the  interpolative  model  without 
lines,  but  not  as  well  as  the  interpolative  model  with  straight  line  constraints.  The  interpolative  model 
without  straight  line  constraints  degrades  more  rapidly  than  the  polynomial  model  as  the  amount  of  con¬ 
trol  is  decreased  (going  from  the  heavy  (1)  to  the  sparse  (3)  point  densities).  This  is  particularly  evident  in 
the  maximum  error  statistic  in  the  Y  direction. 

In  nearly  all  cases,  the  error  in  the  Y  direction  (approximately  cross-strip)  is  worse  than  in  the  X  direc¬ 
tion  (approximately  along-strip).  A  possible  explanation  for  this  is  uncompensated  sensor  roll,  due  to  a 
combination  of  atmospheric  turbulence  and  a  malfunctioning  stabilization  platform. 

Effectiveness  of  straight  line  constraints.  The  inclusion  of  straight  line  constraints  in  the  interpola¬ 
tive  model  solutions  improved  the  results  in  every  case.  While  decreasing  the  number  of  tie  points  still 
increased  check-point  error,  the  results  from  the  runs  with  sparse  points  (case  3)  are  still  better  than  the 
results  for  the  heavy  point  density  (case  1)  without  lines.  This  indicates  that  straight  line  constraints  can 
be  used  both  to  improve  a  solution  or  as  an  effective  substitute  for  additional  tie  points;  however,  adding 
the  straight  line  constraints  to  the  polynomial  model  solution  made  only  negligible  differences.  It  may  be 
that  the  polynomial  model,  with  its  more  limited  flexibility,  is  unable  to  use  the  additional  information 
from  the  line  constraints. 

It  is  interesting  to  note  the  improvement  in  the  X  direction  from  the  addition  of  the  lines.  The  lines 
are  selected  parallel  to  the  flight  direction  and  would  be  expected  to  mostly  improve  positioning  in  the  Y 
(cross-strip)  direction.  It  is  possible  that  since  the  lines  make  an  approximate  30-degree  angle  to  the  flight 
direction,  they  add  some  geometric  strength  in  the  X  direction. 

Reference  line  spacing  Decreasing  the  reference  line  spacing  for  the  interpolative  model  will  make 
the  model  more  flexible  by  increasing  its  degrees  of  freedom.  Given  enough  information  to  determine 
the  model,  it  should  recreate  the  platform  motion  more  accurately  and  give  better  results.  In  this  case, 
however,  decreasing  the  reference  line  spacing  generally  degraded  the  results.  The  additional  degrees  of 
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(a)  EADT9  test  area  from  Flightline  4  (b)  CHAFFEE  test  area  from  Flightline  7 

Figure  5:  Simulated  Daedalus  ATM  near  infrared  imagery  (Band  7)  of  classification  test  areas. 


freedom  were  not  adequately  determined  by  the  available  information,  and,  in  fact,  the  solution  using  the 
sparse  point  density  without  lines  did  not  converge. 

Variation  between  images.  As  mentioned  above,  the  characteristics  of  the  platform  motion  can  change 
drastically  during  a  mission.  The  two  HYDICE  images  used  in  this  experiment  demonstrate  this;  exam¬ 
ination  of  Figures  1(a)  and  1(b)  show  that  image  5.3  is  much  more  deformed  than  image  4_3.  Statistical 
evidence  of  this  is  given  in  Table  3  for  a  few  selected  test  cases  (interpolative  model  with  line  constraints). 
Note  that  the  median  deviations  for  the  two  images  are  very  comparable  for  each  case,  but  the  maximum 
check-point  deviations  are  much  larger  for  image  5_3  than  for  4_3. 

Figure  4  shows  the  check-point  errors  for  the  heavy  (1)  and  sparse  (3)  point  densities  (interpolative 
model  using  line  constraints,  64  reference  line  spacing).  Note  that  check-points  that  appear  on  both  HY¬ 
DICE  images  have  two  error  vectors,  since  they  are  treated  as  independent  points  in  each  image.  The 
sparse  case  shows  much  larger  check-point  errors  for  a  few  points;  the  largest  for  the  sparse  case  is  56.6 
meters,  but  21.7  meters  for  the  heavy  case.  In  both  cases,  most  points  have  relatively  small  errors,  with 
the  larger  errors  occurring  in  groups.  These  groupings  tend  to  indicate  areas  of  higher  image  deformation 
or  weaknesses  in  the  control  configuration. 


3.  Automated  Analysis  of  HYDICE  imagery 

Because  of  the  volume  of  image  data  collected  by  the  HYDICE  hyperspectral  sensor,  these  classification 
experiments  used  a  reduced  image  dataset.  To  build  on  our  previous  experience  with  Daedalus  ATM 
imagery,  we  simulated  Daedalus  ATM  imagery  by  averaging  the  HYDICE  imagery  bands  contained  within 
the  solar  reflective  bandpasses  of  the  Daedalus  ATM  scanner  (Table  4). 

Figure  5  shows  the  two  test  areas  used  in  the  surface  material  classification  experiments.  The  test  areas’ 
scene  composition  include  motor  pool/barracks  (Figure  5(a))  and  residential  (Figure  5(b))  landscapes. 
Differing  percentages  and  types  of  natural  and  man-made  materials  are  present  in  each  of  these  test  areas. 

The  test  areas  are  from  Flightlines  4  and  7,  which  have  acquisition  time  differences  of  60  minutes.  Down- 
welling  radiometric  conditions  between  these  flightlines  changed  significantly,  as  recorded  by  ground  spec- 


Table  4:  Daedalus  ATM  spectral  bandpasses. 


Band 

Number 

Bandpass 

(micrometer) 

Band 

Number 

Bandpass 

(micrometer) 

1 

0.420  -  0.450 

6 

0.690  -  0.750 

0.450  -  0.520 

7 

0.760  -  0.900 

0.520  -  0.600 

8 

0.910  -  1.050 

0.600  -  0.620 

9 

1.550  -  1.750 

0.630  -  0.690 

2.080  -  2.350 

(a)  RADT9  man-made 
surface/roofing 


(b)  RADT9  bare  earth/  (c)  RADT9  vegetation/ water 

shadow 


(d)  CHAFFEE  man-made 
surface/roofing 


(e)  CHAFFEE  bare  earth/ 
shadow 


(f)  CHAFFEE  vegetation/ 
water 


H  asphalt  lip  concrete  Unew  asphalt  roofing! 

■  HI 

■  deciduous  tree 

T 

deep  water  J 

Bold  asphalt  roofing  §  sheet  metal  roofing  unclassified  1 

i—- 

shadow  I  unclassified 

- 

Jl:  turbid  water 

mm 

J 

unclassified  I 

Figure  6:  Test  area  subsection  surface  material  classification. 

tral  radiance  measurements  during  the  overflights.  To  minimize  the  effects  of  changing  solar  illumination 
and  atmospheric  conditions  between  flightlines,  the  simulated  Daedalus  ATM  imagery  was  converted  to  ap¬ 
parent  reflectance  by  using  the  gray-scale  panel  imagery  and  spectral  reflectance  measurements  to  calculate 
band  gain  and  offset  coefficients  for  each  flightline. 

Manually-selected  training  sets  for  the  materials  listed  in  the  “Fine  Surface  Material”  column  of  Table  5 
were  compiled  from  an  earlier  section  of  Flightline  4.  A  Gaussian  Maximum  Likelihood  (GML)  classifica¬ 
tion  was  performed  using  the  10  simulated  Daedalus  ATM  bands  and  selected  training  sets.  Figure  6  shows 
a  surface  material  subsection  map  from  the  resulting  classification.  These  subsection  maps  correspond  to 
the  outlined  regions  shown  in  Figure  5  for  the  respective  test  areas. 

The  resulting  surface  material  maps  were  evaluated  against  manually-generated  surface  material  ground 
truths  for  each  test  area.  Classification  accuracies  are  57.9  percent  for  RADT9  and  60.4  percent  for  CHAFFEE. 
From  Table  6,  almost  20  percent  of  RADT9’s  classification  error  is  associated  with  confusion  among  con¬ 
crete,  asphalt,  soil  and  gravel.  Looking  at  Figure  6(a),  there  is  breakup  of  the  parking  lot  into  asphalt  and 
concrete  sections  probably  influenced  by  surface  weathering  and  vehicular  traffic.  Also,  the  barrack  roofs 
fluctuate  in  surface  material  classification  due  to  illumination  changes  influenced  by  building  roof  struc- 
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Table  5:  Fine-to-coarse  class  grouping. 


Coarse 

Fine 

Surface  Material 

Surface  Material 

man-made  surface 

asphalt 

concrete 

soil 

bare  earth 

clay 

gravel 

vegetation 

grass 

deciduous  tree 
coniferous  tree 

water 

deep  water 
turbid  water 

man-made  roofing 

new  asphalt  roofing 
old  asphalt  roofing 
sheet  metal  roofing 

shadow 

shadow 

Table  6:  RADT9  top  5  confusion  pairs. 


Ground  Truth 

Classification 

Number 

Error 

Class 

Class 

Confused 

Percentage 

concrete 

asphalt 

7074 

hhhi 

soil 

gravel 

2756 

HE MSI 

grass 

soil 

3.3% 

asphalt 

soil 

~2 Wo 

concrete 

soil 

1558 

2 Wo 

15324 

22.4% 

Table  7:  chaffee  top  5  confusion  pairs. 


Ground  Truth 

Classification 

Number 

Error 

Class 

Class 

Confused 

Percentage 

shadow 

deciduous  tree 

3571 

4.8% 

deciduous  tree 

grass 

■EmM 

4J% 

old  asphalt 
roofing 

concrete 

3.4% 

grass 

deciduous  tree 

2084 

2 Wo 

asphalt 

soil 

■ESSE 

2.2% 

Total 

■EMB 

17.9% 

ture.  Reviewing  Table  7,  approximately  12  percent  of  Chaffee’s  classification  error  belongs  to  shadow, 
deciduous  tree  and  grass  confusions.  From  Figure  5(b)  and  Figure  6(f),  trees  and  grass  dominate  the  scene 
content  of  this  test  area.  Some  of  this  reported  error  is  inherent  in  the  ground  truth  due  to  the  complexity 
of  attempting  to  segment  out  regions  of  overlapping  tree  shadows  and  canopies  surrounded  by  grass. 

We  also  are  interested  in  coarse  surface  material  classification,  whereby  the  fine  surface  material  classes 
are  grouped  into  more  general  categories  as  listed  in  Table  5.  This  type  of  broad  categorization  is  useful 
in  identifying  areas  containing  man-made  or  natural  surface  features.  Table  8  and  Table  9  display  the 


Table  8:  RADT9  coarse  classification  error  matrix. 


TEST 

REFERENCE  — | 

man-made 

surface 

bare 

earth 

vegetation 

water 

man-made 

roofing 

shadow 

Row 

Total 

Commission 

Error 

26677 

1973 

1509 

0 

1477 

32244 

17.3 

|■■2!!EE£EQ!1IIH 

5129 

7299 

2444 

0 

254 

58 

15184 

51.9 

vegetation 

377 

822 

14666 

0 

117 

149 

16131 

9.1 

water 

0 

0 

0 

0 

0 

* 

man-made  roofing 

671 

182 

285 

0 

1598 

Ht« 

48.5 

shadow 

■HuSHU 

33 

425 

0 

79 

36.6 

Column  Total 

10309 

0 

2335 

tmm 

Omission  Error 

17.3 

51.9 

9.1 

* 

48.5 

36.6 

— 

Percent 

|  Overall  Accuracy  =  5; 

L393  /  68482  =  75.0%  j 

Table  9:  CHAFFEE  coarse  classification  error  matrix. 


TEST 

REFEl 

'iENCE  1 

man-made 

surface 

bare 

earth 

vegetation 

water 

man-made 

roofing 

shadow 

man-made  surface 

6693 

1773 

3809 

921 

mom 

49.3 

bare  earth 

■KmSi 

1891 

0 

684 

548 

■tewtfaa 

vegetation 

472 

0 

644 

5986 

13.9 

water 

0 

0 

0 

0 

0 

* 

man-made  roofing 

80 

0 

92 

0 

564 

116  ! 

852 

33.8 

77 

496 

318 

3418 

26.1 

9535 

0 

48201 

0 

6019 

IlIiMiEM 

73853 

Omission  Error 

■HESS 

* 

13.9 

* 

33.8 

26.1 

|  Overall  Accuracy  =  53733  /  73853  =  72.8% 

error  matrices  for  the  coarse  classification  for  each  of  the  test  areas.  Coarse  classification  accuracies  range 
from  72.8  percent  (chaffee)  to  75.0  percent  (radt9).  For  RADT9,  the  majority  of  the  error  (10.4  percent) 
involves  man-made  surface  and  bare  earth  confusions  while  Chaffee’s  dominant  error  (8.1  percent)  is 
confusion  between  shadow  and  vegetation.  These  coarse  surface  material  confusions  have  similar  trends 
that  were  seen  in  the  earlier  fine  material  classification. 

t  We  are  currently  working  on  the  use  of  HYDICE’s  high  spectral  resolution  for  spectral  similarity  and 
linear  mixture  analysis  with  ground-measured  surface  material  reflectance  curves. 


4.  Conclusions 

This  work  has  shown  the  effectiveness  of  straight-line  constraints  in  the  block  adjustment  of  linear  push- 
broom  imagery  when  used  in  conjunction  with  an  interpolative  platform  model.  The  techniques  demon¬ 
strated  can  be  used  in  the  absence  of  navigation  data,  as  was  the  case  for  the  HYDICE  imagery,  or  in  con¬ 
junction  with  navigation  data  in  order  to  improve  the  accuracy  and  reliability  of  the  positioning  solution. 
This  work  will  be  applied  to  the  adjustment  of  our  full  block  of  HYDICE  imagery,  now  in  progress. 

Despite  the  lack  of  navigation  information  in  the  current  investigation,  there  is  no  theoretical  or  practical 
reason  not  to  use  these  techniques  in  conjunction  with  GPS/INS  data;  indeed,  our  initial  plan  was  to 
include  navigation  information  in  the  block  adjustment  solution.  Unless  the  navigation  sensor  accuracy 
and  the  system  calibration  are  of  very  high  order,  or  unless  a  very  large  number  of  high  quality  (possibly 
automatically  generated)  tie  points  are  available,  it  would  appear  that  the  use  of  straight  line  constraints 
can  make  significant  contributions  to  the  accuracy  of  a  block  adjustment. 
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Our  ongoing  research  will  attempt  to  establish  whether  additional  improvements  in  accuracy  can  be 
obtained  or  whether  we  are  at  the  accuracy  limits  of  the  sensor/ platform  combination  without  the  addition 
of  further  information.  Possible  avenues  of  investigation  will  include  increasing  the  number  of  tie  points 
by  using  automated  measurement  techniques  [Heipke  et  al. ,  1996]  and  experimentation  with  other  types  of 
geometric  constraints,  such  as  right  angles. 

With  the  acquisition  of  a  high  spatial  resolution  HYDICE  image  set,  an  opportunity  exists  to  exploit 
the  spectral  information  from  hyperspectral  imagery  to  aid  urban  scene  analysis  for  cartographic  feature 
extraction.  Radiometric  effects  from  changing  solar  illumination  and  atmospheric  conditions  must  be 
accounted  for  in  order  to  support  comparison  of  surface  material  properties  across  HYDICE  flightlines. 

To  use  the  surface  material  information  to  its  full  potential,  accurate  geopositioning  of  HYDICE  imagery  is 
crucial  for  fusing  surface  material  information  with  stereo  and  monocular  cues. 
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